{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "34a4c60d-fc51-494f-91f2-b90815ad05d4",
   "metadata": {},
   "source": [
    "# Goldman Sachs Marquee Security Master"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "22e2f1c2-e053-4f21-849d-ba7f88ea8667",
   "metadata": {
    "tags": []
   },
   "source": [
    "## Introduction\n",
    "This notebook provides examples using the GS-Quant Security Master SDK.  The examples  will help users resolve current identifiers to historical identifiers, retrieve detailed security data, and perform various queries. <br><br>\n",
    "**Security Master developer documentation is available here:**\n",
    "[https://developer.gs.com/p/docs/services/data/securities-master/](https://developer.gs.com/p/docs/services/data/securities-master/)<br><br>\n",
    "**GS-Quant SDK documentation is available here:**\n",
    "[https://developer.gs.com/docs/gsquant/data/accessing-data/secmaster_sdk/](https://developer.gs.com/docs/gsquant/data/accessing-data/secmaster_sdk/)<br><br>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "695687a2-dce9-4d4d-bd90-cf90b98d19c7",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import datetime as dt\n",
    "\n",
    "from IPython.display import Markdown\n",
    "from pprint import pprint\n",
    "from gs_quant.target.secmaster import SecMasterAssetType\n",
    "from gs_quant.api.gs.secmaster import GsSecurityMasterApi\n",
    "from gs_quant.markets.securities import SecurityIdentifier\n",
    "from gs_quant.session import GsSession\n",
    "\n",
    "GsSession.use(client_id=\"INSERT HERE\", client_secret=\"INSERT HERE\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b1f49cfd-2469-4ab5-bc3f-e41266b2d8e1",
   "metadata": {},
   "source": [
    "## Sec Master ID Sorter\n",
    "In the following example is a function using the Get Many Securities method where the user can provide any identifier and have them resolved to an immutable Sec Master ID ('id') along with other market standard symbology."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "48a840db-28f4-4d9e-ac89-e6930b92727a",
   "metadata": {},
   "outputs": [],
   "source": [
    "def sec_master_id_sorter(identifiers):\n",
    "    \"\"\"\n",
    "    Fetch security data and normalize the identifiers.\n",
    "\n",
    "    :param identifiers: List of security identifiers\n",
    "    :return: DataFrame containing normalized identifiers\n",
    "    \"\"\"\n",
    "    data = GsSecurityMasterApi.get_many_securities(identifier=identifiers, isPrimary=True)\n",
    "    results = pd.DataFrame(data['results'])\n",
    "    return pd.json_normalize(results['identifiers'])\n",
    "\n",
    "\n",
    "# Example usage\n",
    "identifiers = [\"GOOGL UW\", \"VOD.N\", \"US67066G1040\"]\n",
    "normalized_identifiers = sec_master_id_sorter(identifiers)\n",
    "pd.DataFrame(normalized_identifiers)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2eb80c58-12e4-41bc-aa05-cdf96a1f5de8",
   "metadata": {},
   "source": [
    "## Map Current Symbology to Historical Symbology\n",
    "The following example is a function using the Get Many Securities method that resolves current symbology to historical symbology, ensuring that the user has correct identifiers as of a given point in time."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "adb107a1-493c-45c2-af05-070ab40871fa",
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_historical_ids(identifiers, date, listed=True, id_type='bbid'):\n",
    "    \"\"\"\n",
    "    Fetch historical identifiers for multiple securities.\n",
    "\n",
    "    :param identifiers: List of current identifiers of the securities.\n",
    "    :param date: The date for which the historical identifiers are required.\n",
    "    :param listed: Whether to filter by listed securities.\n",
    "    :param id_type: The type of identifier to retrieve.\n",
    "    :return: Dictionary mapping each identifier to its historical identifier.\n",
    "    \"\"\"\n",
    "    historical_ids = {}\n",
    "    for identifier in identifiers:\n",
    "        try:\n",
    "            data = GsSecurityMasterApi.get_many_securities(identifier=[identifier], isPrimary=True)\n",
    "            if not data.get('results'):\n",
    "                raise ValueError(f\"No security found for identifier {identifier}\")\n",
    "\n",
    "            secm_id = data['results'][0]['id']\n",
    "            historical_data = GsSecurityMasterApi.get_identifiers(secmaster_id=secm_id)\n",
    "\n",
    "            match = next(\n",
    "                (\n",
    "                    record['value']\n",
    "                    for record in historical_data\n",
    "                    if dt.datetime.strptime(record['startDate'], '%Y-%m-%d').date()\n",
    "                    <= date\n",
    "                    <= (\n",
    "                        dt.datetime.strptime(record['endDate'], '%Y-%m-%d').date()\n",
    "                        if record['endDate'] != '9999-99-99'\n",
    "                        else dt.date.today()\n",
    "                    )\n",
    "                    and (not listed or record['type'] == id_type)\n",
    "                ),\n",
    "                None,\n",
    "            )\n",
    "            historical_ids[identifier] = match or identifier\n",
    "        except Exception as e:\n",
    "            print(f\"Error fetching historical ID for {identifier}: {e}\")\n",
    "            historical_ids[identifier] = None\n",
    "    return historical_ids\n",
    "\n",
    "\n",
    "# Example usage\n",
    "identifiers = [\"META UW\", \"GOOG UW\", \"XYZ UN\"]\n",
    "query_date = dt.date(2017, 1, 1)\n",
    "\n",
    "try:\n",
    "    historical_ids = get_historical_ids(identifiers, query_date)\n",
    "    for identifier, historical_id in historical_ids.items():\n",
    "        print(f\"Historical ID for {identifier} as of {query_date}: {historical_id}\")\n",
    "except ValueError as e:\n",
    "    print(f\"Error: {e}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bfd14a6a-7927-4f67-bda5-7b67f2ae699e",
   "metadata": {},
   "source": [
    "## Query by Identifier\n",
    "This example allows the user to quickly retrieve the latest available information about a specific security."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "263c5bf5-9338-4c11-a6a8-cd7c28c778b4",
   "metadata": {},
   "outputs": [],
   "source": [
    "data = GsSecurityMasterApi.get_security_data(id_value=\"GS UN\", id_type=SecurityIdentifier.BBID)\n",
    "pprint(data)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8c4a68e7-5e14-4dbb-8d35-8162a32d8e21",
   "metadata": {},
   "source": [
    "## Query by Attribute\n",
    "This example allows the user to specify an asset type or set of identifiers to retrieve matching securities."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3c01274f-20b5-48f2-a050-3068ce3e0b8c",
   "metadata": {},
   "outputs": [],
   "source": [
    "data = GsSecurityMasterApi.get_many_securities(type_=SecMasterAssetType.Common_Stock, limit=1)\n",
    "results = data['results']\n",
    "pprint(results)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e398e263-5c5c-4517-9617-1d9e36ffd731",
   "metadata": {},
   "source": [
    "## Get All Securities\n",
    "This example is an expansive layer over 'Query by Attribute' and is tailored for retrieving an extensive list of assets within a specific type.\n",
    "It simplifies the user experience by automatically handling the scrolling.\n",
    "The output below shows the first two results for Dutch Certificates.  This example was selected to optimize speed since there are comparably few assets of this type."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e0e00289-a159-43a6-a4aa-e288f7934d49",
   "metadata": {},
   "outputs": [],
   "source": [
    "data = GsSecurityMasterApi.get_all_securities(type_=SecMasterAssetType.Dutch_Cert)\n",
    "pprint(data['results'][0:2])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c3b9d49e-5c55-4548-b696-277bcb7f0a94",
   "metadata": {},
   "source": [
    "## Get Identifier History\n",
    "This example provides a chronological account of how a particular identifier has changed for a security over time.\n",
    "This method only accepts 'id' as an input, which is the immutable Sec Master ID."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cc3acf88-109f-4a61-959c-e31fe78055a8",
   "metadata": {},
   "outputs": [],
   "source": [
    "secm_id = \"GSPD227284E459\"\n",
    "pd.DataFrame(GsSecurityMasterApi.get_identifiers(secmaster_id=secm_id))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fb699965-a231-476d-ae8e-2de27b0dee77",
   "metadata": {},
   "source": [
    "## Map Securities\n",
    "This example converts between different types of security identifiers and accounts for historical changes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0391f211-7631-4d46-bcb5-241dcd5925ca",
   "metadata": {},
   "outputs": [],
   "source": [
    "pd.DataFrame(\n",
    "    GsSecurityMasterApi.map(\n",
    "        SecurityIdentifier.GSID,\n",
    "        [\"40602012\"],\n",
    "        start_date=\"2007-01-01\",\n",
    "        end_date=\"2024-10-29\",\n",
    "        output_types=[SecurityIdentifier.BBID, SecurityIdentifier.CUSIP, SecurityIdentifier.SEDOL],\n",
    "    )\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "29a1a54c-9286-41b0-afd6-a2eda9d49ebf",
   "metadata": {},
   "source": [
    "## Search Securities\n",
    "This example allows for full-text searches across many fields, including identifiers and company names."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5cf5fc0b-53c7-478c-9f6c-55d6331edff8",
   "metadata": {},
   "outputs": [],
   "source": [
    "results = GsSecurityMasterApi.search(\n",
    "    q=\"TESLA\", is_primary=True, active_listing=True, type_=SecMasterAssetType.Common_Stock\n",
    ")\n",
    "[\n",
    "    f\"The bbid is: {data['identifiers']['bbid']} and the issuer name is: {data['issuer']['name']}  and the type is: {data['type']}.\"\n",
    "    for data in results\n",
    "]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6cc58f33-3768-4c4f-b6d7-15594f4e8c9c",
   "metadata": {},
   "source": [
    "## Deltas - Get Recent Changes\n",
    "This example retrieves the list of recent identifier changes within a specified time range."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "29454b40-bc16-444b-9541-3dfa1596c6af",
   "metadata": {},
   "outputs": [],
   "source": [
    "start_time = dt.datetime.now() - dt.timedelta(hours=1)\n",
    "data = GsSecurityMasterApi.get_deltas(raw=False, start_time=start_time)\n",
    "pd.DataFrame(data['results'])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "450cac0b-7360-4a83-a366-0b66bb412248",
   "metadata": {},
   "source": [
    "## Issuers - Capital Structure\n",
    "This example provides all listed securities types for a given issuer. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "54f6356a-a632-4357-b451-e9bb20ac76ac",
   "metadata": {},
   "outputs": [],
   "source": [
    "bbid = \"META UW\"\n",
    "data = GsSecurityMasterApi.get_capital_structure(id_value=bbid, id_type=SecurityIdentifier.BBID)\n",
    "res = data['results'][0]\n",
    "display(Markdown(f\"Capital structure for {bbid} matched {res['issuerName']}\"))\n",
    "display(pd.DataFrame([data['assetTypesTotal']]))\n",
    "for t in res['types']:\n",
    "    display(Markdown(f\"**{t}**\"))\n",
    "    display(pd.DataFrame(res['types'][t][:5]))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7351a972-cc65-4387-b18b-a3e210a104dd",
   "metadata": {},
   "source": [
    "## Exchanges\n",
    "This example retrieves detailed information about exchanges."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9b5df6a8-547b-4c2b-bbf4-8064b8433e75",
   "metadata": {},
   "outputs": [],
   "source": [
    "data = GsSecurityMasterApi.get_exchanges()\n",
    "pd.DataFrame(data['results'][:5])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4986c8c5-d0ee-436e-ad08-919b72721a05",
   "metadata": {},
   "source": [
    "### Exchanges - Query by Exchange Code\n",
    "This example retrieves detailed information about exchanges by selected exchange codes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1ea8dcb6-5f57-47ca-b20a-7bc59f87c9e2",
   "metadata": {},
   "outputs": [],
   "source": [
    "data = GsSecurityMasterApi.get_exchanges(mic=\"XBUE\")\n",
    "pd.DataFrame(data['results'])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ef9b840f-0688-489e-b207-cf4327f584bc",
   "metadata": {},
   "source": [
    "### Exchanges - Get Exchange Code History\n",
    "This example provides the history of exchange codes.\n",
    "This method only accepts gsExchangeId as an input."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2106a6f0-8578-458c-83cf-73af2500ccf0",
   "metadata": {},
   "outputs": [],
   "source": [
    "results = GsSecurityMasterApi.get_exchange_identifiers_history(gs_exchange_id=2)\n",
    "pd.DataFrame(results)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5ae9dd3a3e0d75ac",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "## Listed Derivatives by Underlyer\n",
    "This example provides the listed derivatives for a given underlying security.\n",
    "This method only accepts 'id' as an input, which is the immutable Sec Master ID."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cf9bfbdf004b8c63",
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "underlyers = GsSecurityMasterApi.get_securities_by_underlyers(\n",
    "    id_value=\"GSPD100E0\", limit=5, type_=SecMasterAssetType.Equity_Option\n",
    ")\n",
    "print(f\"Offset key is: {underlyers.get('offsetKey', 'No offset key returned')}.\")\n",
    "display(pd.DataFrame(underlyers['results']))\n",
    "\n",
    "if 'offsetKey' in underlyers:\n",
    "    underlyers = GsSecurityMasterApi.get_securities_by_underlyers(\n",
    "        id_value=\"GSPD100E0\", limit=5, type_=SecMasterAssetType.Equity_Option, offset_key=underlyers['offsetKey']\n",
    "    )\n",
    "    print(f\"Offset key is: {underlyers.get('offsetKey', 'No offset key returned')}.\")\n",
    "    display(pd.DataFrame(underlyers['results']))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ae4b7b1c7bf29b1a",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "### Listed Derivatives by Underlyer - fetch all matching securities\n",
    "This example gets all matching listed derivatives"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2043ebbfaf7ddff7",
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "underlyers = GsSecurityMasterApi.get_securities_by_underlyers(\n",
    "    id_value=\"GSPD100E0\", type_=SecMasterAssetType.Equity_Option, scroll_all_pages=True\n",
    ")\n",
    "print(f\"Total assets count is: {underlyers.get('totalResults', 'No totalResults returned')}.\")\n",
    "display(pd.DataFrame(underlyers['results']))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f7705fd742bf8278",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "### Listed Derivatives by Underlyer - Currency and Country Code Filter\n",
    "This example filters listed derivatives by currency and country code.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "aeb1d0666440513c",
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "underlyers = GsSecurityMasterApi.get_securities_by_underlyers(\n",
    "    id_value=\"GSPD100E0\", limit=5, type_=SecMasterAssetType.Equity_Option, currency='USD', country_code='US'\n",
    ")\n",
    "\n",
    "display(pd.DataFrame(underlyers['results']))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "935b7da8ea4c1b50",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "### Listed Derviatives by Underlyer - Effective Date Filter\n",
    "This example filters listed derivatives by effective date."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "753dc7694abe28ff",
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "underlyers = GsSecurityMasterApi.get_securities_by_underlyers(\n",
    "    id_value=\"GSPD100E0\",\n",
    "    limit=5,\n",
    "    type_=SecMasterAssetType.Equity_Option,\n",
    "    effective_date='2023-01-01',\n",
    "    country_code='US',\n",
    ")\n",
    "display(pd.DataFrame(underlyers['results']))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3bf6098b8a896034",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "### Listed Derivatives by Underlyer - Inactive Filter\n",
    "This example includes inactive listed derivatives in the results."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "599c4f6601e35096",
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "underlyers = GsSecurityMasterApi.get_securities_by_underlyers(\n",
    "    id_value=\"GSPD100E0\", limit=5, type_=SecMasterAssetType.Equity_Option, include_inactive=True, country_code='US'\n",
    ")\n",
    "display(pd.DataFrame(underlyers['results']))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e59a4ce1bcdf63d1",
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
